Security systems are finding an ever increasing usage in monitoring installations. Such systems can range from one or two cameras in a small store up to dozens of cameras covering a large mall or building. In general these systems display the video signals as discrete individual pictures on a number of display panels. When there are a large number of cameras, greater than the number of display panels, the systems have a control means that changes the input signal to the displays so as to rotate the images and scan the entire video coverage within a predetermined time frame. Such systems also usually have means to stop the progression of the image sequence to allow study of a particular area of interest. Such systems have proved useful in monitoring areas and frequently result in the identification of criminal activity.
The use of video cameras in such security and surveillance systems typically involves some form of video image processing. One type of image processing methodology involves image fusion, which is a process of combining images, obtained by sensors of different wavelengths simultaneously viewing of the same scene, to form a composite image. The composite image is formed to improve image content and to make it easier for the user to detect, recognize and identify targets and increase his or her situational awareness.
A specific type of image fusion is multi-spectral fusion, which is a process of combining data from multiple sensors operating at different spectral bands (e.g., visible, near infrared, long-wave, infrared, etc.) to generate a single composite image, which contains a complete, accurate and robust description of the scene than any of the individual sensor images.
Current automated (e.g., computerized) video surveillance systems, particularly those involving the use of only video cameras, are plagued by a number of problems. Such video surveillance systems typically generate high false alarm rates, and generally only function well under a narrow range of operational parameters. Most applications, however, especially those that take place outdoors, require a wide range of operation and this causes current surveillance systems to fail due to high false alarm rates and/or frequent misses of an object of interest.
The operator is then forced to turn the system off, because the system in effect cannot be “trusted” to generate reliable data. Another problem inherent with current video surveillance systems is that such systems are severely affected by lighting conditions and weather. Future surveillance systems must be able to operate in a 24 hours, 7 day continuous mode. Most security systems operating during the night are not well lit or all located in situations in which no lighting is present at all. Video surveillance systems must be hardened against a wide range of weather conditions (e.g., rain, snow, dust, hail, etc.).
The objective of performing multi-sensor fusion is to intelligently combine multi-modality sensor imagery, so that a single view of a scene can be provided with extended information content, and enhanced quality video for the operator or user. A number of technical barriers exist, however, to achieving this goal. For example, “pixel level weighted averaging” takes the weighted average of the pixel intensity of varying source images. The technical problem of simple weighted average of pixel intensity is that such a methodology does not consider different environmental conditions.